Building management system performance index

ABSTRACT

One embodiment of the present disclosure is a system. The system includes one or more memory devices having instructions stored thereon that, when executed by one or more processors, cause the one or more processors to perform operations. The operations include: obtaining first data indicating operating parameters of a server of a building management system (BMS), obtaining second data regarding indicating operating parameters of at least one of a supervisory controller or a field controller of the BMS, obtaining third data regarding indicating operating parameters of one or more edge building devices, and calculating a performance index value for the BMS based on an aggregate of the first data, the second data, and the third data.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application claims priority to and the benefit of Indian Provisional Patent Application No. 202021055148, filed Dec. 18, 2020, which is incorporated herein by reference in its entirety.

BACKGROUND

The present disclosure relates generally to building management systems (BMSs) and, more specifically, to determining a performance index for BMSs (e.g., indicative of the overall health and efficiency of a BMS)

In various implementations, a BMS operates by monitoring and controlling a wide variety building subsystems and equipment. A BMS can improve building operations, and can allow building owners or operators to meeting various operating goals, by increasing building (e.g., system and equipment) efficiency, decreasing operating costs, reducing user input (e.g., through automation), reducing downtime, etc. However, problems with a BMS can cause the BMS to operate less efficiently over time. Additionally, it may not be apparent how a BMS is actually performing. Therefore, it would be desirable to provide a mechanism for quantifying the performance of a BMS and identifying when there are issues causing a BMS to perform less than optimally.

SUMMARY

One embodiment of the present disclosure is a system. The system includes one or more memory devices having instructions stored thereon that, when executed by one or more processors, cause the one or more processors to perform operations. The operations include: obtaining first data indicating operating parameters of a server of a building management system (BMS), obtaining second data regarding indicating operating parameters of at least one of a supervisory controller or a field controller of the BMS, obtaining third data regarding indicating operating parameters of one or more edge building devices, and calculating a performance index value for the BMS based on an aggregate of the first data, the second data, and the third data.

Another embodiment of the present disclosure is a method. The method includes: obtaining first data indicating operating parameters of a server of a building management system (BMS), obtaining second data regarding indicating operating parameters of at least one of a supervisory controller or a field controller of the BMS, obtaining third data regarding indicating operating parameters of one or more edge building devices, and calculating a performance index value for the BMS based on an aggregate of the first data, the second data, and the third data.

Yet another embodiment of the present disclosure is non-transitory computer-readable media. The non-transitory computer-readable media includes computer-readable instructions stored thereon that when executed by a processor cause the processor to perform operations. The operations include: obtaining first data indicating operating parameters of a server of a building management system (BMS), obtaining second data regarding indicating operating parameters of at least one of a supervisory controller or a field controller of the BMS, obtaining third data regarding indicating operating parameters of one or more edge building devices, and calculating a performance index value for the BMS based on an aggregate of the first data, the second data, and the third data.

BRIEF DESCRIPTION OF THE DRAWINGS

Various objects, aspects, features, and advantages of the disclosure will become more apparent and better understood by referring to the detailed description taken in conjunction with the accompanying drawings, in which like reference characters identify corresponding elements throughout. In the drawings, like reference numbers generally indicate identical, functionally similar, and/or structurally similar elements.

FIG. 1 is a drawing of a building equipped with a HVAC system, according to some embodiments.

FIG. 2 is a block diagram of a waterside system that may be used in conjunction with the building of FIG. 1, according to some embodiments.

FIG. 3 is a block diagram of an airside system that may be used in conjunction with the building of FIG. 1, according to some embodiments.

FIG. 4 is a block diagram of a building management system (BMS) that may be used to monitor and/or control the building of FIG. 1, according to some embodiments.

FIG. 5 is a block diagram of a system for monitoring various components of a BMS and calculating a BMS performance index (BPI) value, according to some embodiments.

FIG. 6 is a block diagram of a BPI tool implemented in the system of FIG. 5, according to some embodiments.

FIG. 7 is a process for calculating a BPI value for a BMS, according to some embodiments.

FIG. 8 is an example graph for calculating the BPI of a BMS, according to some embodiments.

FIGS. 9 and 10 are example interfaces for graphically presenting the BPI of one or more BMSs, according to some embodiments.

DETAILED DESCRIPTION

Over time, a BMS can become less efficient due to out-of-date firmware and software, incompatibility with newer components or systems, manual controls that are left uncorrected, etc., thereby leading to increased operating costs and downtime. For example, out-of-date BMS software and/or firmware can leave the system vulnerable to cybersecurity threats (e.g., viruses, hackers, etc.). Such out-of-date or otherwise inefficient BMSs may not take advantage of the latest algorithms, components, etc., unless the BMS software and firmware is updated, which can be a time-consuming process.

Additionally, users of a BMS (e.g., facility managers, operators, etc.) may implement various changes to a BMS based on building requirements that, if left uninterrupted, may lead to further inefficiencies. For example, a building manager may manually override an equipment setpoint (e.g., temperature, schedule, etc.) or may control various device manually (e.g., opening or closing a valve or damper), which may initially allow the building manager to meet a building requirement. However, such manual BMS control may not allow the BMS to fully optimize operations. In particular, manual user inputs may prevent the BMS from fully automatic operations and optimization, potentially leading to increased energy usage or loss, maintenance, operating costs, etc. Therefore, it would be desirable to monitor a BMS for inefficiencies, and to report these inefficiencies to a user (e.g., a building manager) in an intuitive format, allowing the user to correct deficiencies quickly and easily.

Referring generally to the FIGURES, a system and methods for calculating a BMS performance index (BPI) are shown, according to some embodiments. BPI may be a value indicative of the overall health and efficiency of a BMS. A BPI value can provide numerous insights to a user (e.g., a building manager, a facilities operator, etc.), allowing the user to quickly and easily determine whether the BMS is running as efficiently as possible. In particular, a BPI tool may be configured to receive operating data from a variety of BMS components, calculate a performance score for each of the variety of BMS components, and calculate an aggregate BPI for the BMS based on the various performance scores. Operating data can be obtain from computing devices (e.g., servers) of the BMS, supervisory and field controllers of the BMS, and from various lower-level (i.e., edge) BMS devices, including sensors, actuating devices, points, etc.

A number of different factors may impact the BPI for a particular BMS. In some embodiments, a number of parameters for each component of the BMS may be established, and operating data for each BMS components may be compared to these parameters to calculate performance scores for each component. For example, parameters such as a maximum average memory usage and an allowable temperature range may be defined for supervisory controllers of a BMS. Any controllers that to not meet these parameters (e.g., memory usage is too high) may incur a penalty score that reduces the performance score for the corresponding controller. Performance scores for all of the computing devices (e.g., servers), supervisory and field controllers, and edge devices in a BMS can be aggregated to generate the BPI.

In some embodiments, the BPI may be utilized to automatically generate recommendations for improving BMS performance. For example, it may be determined that out-of-date server software is negatively impacting BMS performance (e.g., lowering the BPI), so BPI tool may recommend (e.g., to a user) that the server software be updated. In some embodiments, the BPI may also be utilized to automatically schedule service or maintenance for various BMS components. For example, maintenance may automatically be scheduled for a stuck actuator device, which also may lower the BPI. Additionally, in some embodiments, the BPI may be presented via various user interfaces, to allow a user to quickly and intuitively view BMS performance and identify problem areas (e.g., or areas of improvement).

Building with Building Systems

Referring now to FIGS. 1-4, an exemplary BMS and HVAC system in which the systems and methods of the present disclosure can be implemented are shown, according to some embodiments. Referring particularly to FIG. 1, a perspective view of a building 10 is shown. Building 10 is served by a BMS. A BMS is, in general, a system of devices configured to control, monitor, and manage equipment in or around a building or building area. A BMS can include, for example, a HVAC system, a security system, a lighting system, a fire safety system, any other system that is capable of managing building functions or devices, or any combination thereof.

The BMS that serves building 10 includes an HVAC system 100. HVAC system 100 can include a plurality of HVAC devices (e.g., heaters, chillers, air handling units, pumps, fans, thermal energy storage, etc.) configured to provide heating, cooling, ventilation, or other services for building 10. For example, HVAC system 100 is shown to include a waterside system 120 and an airside system 130. Waterside system 120 can provide a heated or chilled fluid to an air handling unit of airside system 130. Airside system 130 can use the heated or chilled fluid to heat or cool an airflow provided to building 10. An exemplary waterside system and airside system which can be used in HVAC system 100 are described in greater detail with reference to FIGS. 2-3.

HVAC system 100 is shown to include a chiller 102, a boiler 104, and a rooftop air handling unit (AHU) 106. Waterside system 120 can use boiler 104 and chiller 102 to heat or cool a working fluid (e.g., water, glycol, etc.) and can circulate the working fluid to AHU 106. In various embodiments, the HVAC devices of waterside system 120 can be located in or around building 10 (as shown in FIG. 1) or at an offsite location such as a central plant (e.g., a chiller plant, a steam plant, a heat plant, etc.). The working fluid can be heated in boiler 104 or cooled in chiller 102, depending on whether heating or cooling is required in building 10. Boiler 104 can add heat to the circulated fluid, for example, by burning a combustible material (e.g., natural gas) or using an electric heating element. Chiller 102 can place the circulated fluid in a heat exchange relationship with another fluid (e.g., a refrigerant) in a heat exchanger (e.g., an evaporator) to absorb heat from the circulated fluid. The working fluid from chiller 102 and/or boiler 104 can be transported to AHU 106 via piping 108.

AHU 106 can place the working fluid in a heat exchange relationship with an airflow passing through AHU 106 (e.g., via one or more stages of cooling coils and/or heating coils). The airflow can be, for example, outside air, return air from within building 10, or a combination of both. AHU 106 can transfer heat between the airflow and the working fluid to provide heating or cooling for the airflow. For example, AHU 106 can include one or more fans or blowers configured to pass the airflow over or through a heat exchanger containing the working fluid. The working fluid can then return to chiller 102 or boiler 104 via piping 110.

Airside system 130 can deliver the airflow supplied by AHU 106 (i.e., the supply airflow) to building 10 via air supply ducts 112 and can provide return air from building 10 to AHU 106 via air return ducts 114. In some embodiments, airside system 130 includes multiple variable air volume (VAV) units 116. For example, airside system 130 is shown to include a separate VAV unit 116 on each floor or zone of building 10. VAV units 116 can include dampers or other flow control elements that can be operated to control an amount of the supply airflow provided to individual zones of building 10. In other embodiments, airside system 130 delivers the supply airflow into one or more zones of building 10 (e.g., via supply ducts 112) without using intermediate VAV units 116 or other flow control elements. AHU 106 can include various sensors (e.g., temperature sensors, pressure sensors, etc.) configured to measure attributes of the supply airflow. AHU 106 can receive input from sensors located within AHU 106 and/or within the building zone and can adjust the flow rate, temperature, or other attributes of the supply airflow through AHU 106 to achieve setpoint conditions for the building zone.

In FIG. 2, waterside system 200 is shown as a central plant having a plurality of subplants 202-212. Subplants 202-212 are shown to include a heater subplant 202, a heat recovery chiller subplant 204, a chiller subplant 206, a cooling tower subplant 208, a hot thermal energy storage (TES) subplant 210, and a cold thermal energy storage (TES) subplant 212. Subplants 202-212 consume resources (e.g., water, natural gas, electricity, etc.) from utilities to serve the thermal energy loads (e.g., hot water, cold water, heating, cooling, etc.) of a building or campus. For example, heater subplant 202 may be configured to heat water in a hot water loop 214 that circulates the hot water between heater subplant 202 and building 10. Chiller subplant 206 may be configured to chill water in a cold water loop 216 that circulates the cold water between chiller subplant 206 building 10. Heat recovery chiller subplant 204 may be configured to transfer heat from cold water loop 216 to hot water loop 214 to provide additional heating for the hot water and additional cooling for the cold water. Condenser water loop 218 may absorb heat from the cold water in chiller subplant 206 and reject the absorbed heat in cooling tower subplant 208 or transfer the absorbed heat to hot water loop 214. Hot TES subplant 210 and cold TES subplant 212 may store hot and cold thermal energy, respectively, for subsequent use.

Hot water loop 214 and cold water loop 216 may deliver the heated and/or chilled water to air handlers located on the rooftop of building 10 (e.g., AHU 106) or to individual floors or zones of building 10 (e.g., VAV units 116). The air handlers push air past heat exchangers (e.g., heating coils or cooling coils) through which the water flows to provide heating or cooling for the air. The heated or cooled air may be delivered to individual zones of building 10 to serve the thermal energy loads of building 10. The water then returns to subplants 202-212 to receive further heating or cooling.

Although subplants 202-212 are shown and described as heating and cooling water for circulation to a building, it is understood that any other type of working fluid (e.g., glycol, CO2, etc.) may be used in place of or in addition to water to serve the thermal energy loads. In other embodiments, subplants 202-212 may provide heating and/or cooling directly to the building or campus without requiring an intermediate heat transfer fluid. These and other variations to waterside system 200 are within the teachings of the present invention.

Each of subplants 202-212 may include a variety of equipment configured to facilitate the functions of the subplant. For example, heater subplant 202 is shown to include a plurality of heating elements 220 (e.g., boilers, electric heaters, etc.) configured to add heat to the hot water in hot water loop 214. Heater subplant 202 is also shown to include several pumps 222 and 224 configured to circulate the hot water in hot water loop 214 and to control the flow rate of the hot water through individual heating elements 220. Chiller subplant 206 is shown to include a plurality of chillers 232 configured to remove heat from the cold water in cold water loop 216. Chiller subplant 206 is also shown to include several pumps 234 and 236 configured to circulate the cold water in cold water loop 216 and to control the flow rate of the cold water through individual chillers 232.

Heat recovery chiller subplant 204 is shown to include a plurality of heat recovery heat exchangers 226 (e.g., refrigeration circuits) configured to transfer heat from cold water loop 216 to hot water loop 214. Heat recovery chiller subplant 204 is also shown to include several pumps 228 and 230 configured to circulate the hot water and/or cold water through heat recovery heat exchangers 226 and to control the flow rate of the water through individual heat recovery heat exchangers 226. Cooling tower subplant 208 is shown to include a plurality of cooling towers 238 configured to remove heat from the condenser water in condenser water loop 218. Cooling tower subplant 208 is also shown to include several pumps 240 configured to circulate the condenser water in condenser water loop 218 and to control the flow rate of the condenser water through individual cooling towers 238.

Hot TES subplant 210 is shown to include a hot TES tank 242 configured to store the hot water for later use. Hot TES subplant 210 may also include one or more pumps or valves configured to control the flow rate of the hot water into or out of hot TES tank 242. Cold TES subplant 212 is shown to include cold TES tanks 244 configured to store the cold water for later use. Cold TES subplant 212 may also include one or more pumps or valves configured to control the flow rate of the cold water into or out of cold TES tanks 244.

In some embodiments, one or more of the pumps in waterside system 200 (e.g., pumps 222, 224, 228, 230, 234, 236, and/or 240) or pipelines in waterside system 200 include an isolation valve associated therewith. Isolation valves may be integrated with the pumps or positioned upstream or downstream of the pumps to control the fluid flows in waterside system 200. In various embodiments, waterside system 200 may include more, fewer, or different types of devices and/or subplants based on the particular configuration of waterside system 200 and the types of loads served by waterside system 200.

Referring now to FIG. 3, a block diagram of an airside system 300 is shown, according to some embodiments. In various embodiments, airside system 300 may supplement or replace airside system 130 in HVAC system 100 or may be implemented separate from HVAC system 100. When implemented in HVAC system 100, airside system 300 may include a subset of the HVAC devices in HVAC system 100 (e.g., AHU 106, VAV units 116, ducts 112-114, fans, dampers, etc.) and may be located in or around building 10. Airside system 300 may operate to heat or cool an airflow provided to building 10 using a heated or chilled fluid provided by waterside system 200.

In FIG. 3, airside system 300 is shown to include an economizer-type air handling unit (AHU) 302. Economizer-type AHUs vary the amount of outside air and return air used by the air handling unit for heating or cooling. For example, AHU 302 may receive return air 304 from building zone 306 via return air duct 308 and may deliver supply air 310 to building zone 306 via supply air duct 312. In some embodiments, AHU 302 is a rooftop unit located on the roof of building 10 (e.g., AHU 106 as shown in FIG. 1) or otherwise positioned to receive both return air 304 and outside air 314. AHU 302 may be configured to operate exhaust air damper 316, mixing damper 318, and outside air damper 320 to control an amount of outside air 314 and return air 304 that combine to form supply air 310. Any return air 304 that does not pass through mixing damper 318 may be exhausted from AHU 302 through exhaust damper 316 as exhaust air 322.

Each of dampers 316-320 may be operated by an actuator. For example, exhaust air damper 316 may be operated by actuator 324, mixing damper 318 may be operated by actuator 326, and outside air damper 320 may be operated by actuator 328. Actuators 324-328 may communicate with an AHU controller 330 via a communications link 332. Actuators 324-328 may receive control signals from AHU controller 330 and may provide feedback signals to AHU controller 330. Feedback signals may include, for example, an indication of a current actuator or damper position, an amount of torque or force exerted by the actuator, diagnostic information (e.g., results of diagnostic tests performed by actuators 324-328), status information, commissioning information, configuration settings, calibration data, and/or other types of information or data that may be collected, stored, or used by actuators 324-328. AHU controller 330 may be an economizer controller configured to use one or more control algorithms (e.g., state-based algorithms, extremum seeking control (ESC) algorithms, proportional-integral (PI) control algorithms, proportional-integral-derivative (PID) control algorithms, model predictive control (MPC) algorithms, feedback control algorithms, etc.) to control actuators 324-328.

Still referring to FIG. 3, AHU 302 is shown to include a cooling coil 334, a heating coil 336, and a fan 338 positioned within supply air duct 312. Fan 338 may be configured to force supply air 310 through cooling coil 334 and/or heating coil 336 and provide supply air 310 to building zone 306. AHU controller 330 may communicate with fan 338 via communications link 340 to control a flow rate of supply air 310. In some embodiments, AHU controller 330 controls an amount of heating or cooling applied to supply air 310 by modulating a speed of fan 338.

Cooling coil 334 may receive a chilled fluid from waterside system 200 (e.g., from cold water loop 216) via piping 342 and may return the chilled fluid to waterside system 200 via piping 344. Valve 346 may be positioned along piping 342 or piping 344 to control a flow rate of the chilled fluid through cooling coil 334. In some embodiments, cooling coil 334 includes multiple stages of cooling coils that can be independently activated and deactivated (e.g., by AHU controller 330, by BMS controller 366, etc.) to modulate an amount of cooling applied to supply air 310.

Heating coil 336 may receive a heated fluid from waterside system 200 (e.g., from hot water loop 214) via piping 348 and may return the heated fluid to waterside system 200 via piping 350. Valve 352 may be positioned along piping 348 or piping 350 to control a flow rate of the heated fluid through heating coil 336. In some embodiments, heating coil 336 includes multiple stages of heating coils that can be independently activated and deactivated (e.g., by AHU controller 330, by BMS controller 366, etc.) to modulate an amount of heating applied to supply air 310.

Each of valves 346 and 352 may be controlled by an actuator. For example, valve 346 may be controlled by actuator 354 and valve 352 may be controlled by actuator 356. Actuators 354-356 may communicate with AHU controller 330 via communications links 358-360. Actuators 354-356 may receive control signals from AHU controller 330 and may provide feedback signals to controller 330. In some embodiments, AHU controller 330 receives a measurement of the supply air temperature from a temperature sensor 362 positioned in supply air duct 312 (e.g., downstream of cooling coil 334 and/or heating coil 336). AHU controller 330 may also receive a measurement of the temperature of building zone 306 from a temperature sensor 364 located in building zone 306.

In some embodiments, AHU controller 330 operates valves 346 and 352 via actuators 354-356 to modulate an amount of heating or cooling provided to supply air 310 (e.g., to achieve a setpoint temperature for supply air 310 or to maintain the temperature of supply air 310 within a setpoint temperature range). The positions of valves 346 and 352 affect the amount of heating or cooling provided to supply air 310 by cooling coil 334 or heating coil 336 and may correlate with the amount of energy consumed to achieve a desired supply air temperature. AHU controller 330 may control the temperature of supply air 310 and/or building zone 306 by activating or deactivating coils 334-336, adjusting a speed of fan 338, or a combination of both.

Still referring to FIG. 3, airside system 300 is shown to include a building automation system (BMS) controller 366 and a client device 368. BMS controller 366 may include one or more computer systems (e.g., servers, supervisory controllers, subsystem controllers, etc.) that serve as system level controllers, application or data servers, head nodes, or master controllers for airside system 300, waterside system 200, HVAC system 100, and/or other controllable systems that serve building 10. BMS controller 366 may communicate with multiple downstream building systems or subsystems (e.g., HVAC system 100, a security system, a lighting system, waterside system 200, etc.) via a communications link 370 according to like or disparate protocols (e.g., LON, BACnet, etc.). In various embodiments, AHU controller 330 and BMS controller 366 may be separate (as shown in FIG. 3) or integrated. In an integrated implementation, AHU controller 330 may be a software module configured for execution by a processor of BMS controller 366.

In some embodiments, AHU controller 330 receives information from BMS controller 366 (e.g., commands, setpoints, operating boundaries, etc.) and provides information to BMS controller 366 (e.g., temperature measurements, valve or actuator positions, operating statuses, diagnostics, etc.). For example, AHU controller 330 may provide BMS controller 366 with temperature measurements from temperature sensors 362-364, equipment on/off states, equipment operating capacities, and/or any other information that can be used by BMS controller 366 to monitor or control a variable state or condition within building zone 306.

Client device 368 may include one or more human-machine interfaces or client interfaces (e.g., graphical user interfaces, reporting interfaces, text-based computer interfaces, client-facing web services, web servers that provide pages to web clients, etc.) for controlling, viewing, or otherwise interacting with HVAC system 100, its subsystems, and/or devices. Client device 368 may be a computer workstation, a client terminal, a remote or local interface, or any other type of user interface device. Client device 368 may be a stationary terminal or a mobile device. For example, client device 368 may be a desktop computer, a computer server with a user interface, a laptop computer, a tablet, a smartphone, a PDA, or any other type of mobile or non-mobile device. Client device 368 may communicate with BMS controller 366 and/or AHU controller 330 via communications link 372.

Referring now to FIG. 4, a block diagram of a building automation system (BMS) 400 is shown, according to some embodiments. BMS 400 may be implemented in building 10 to automatically monitor and control various building functions. BMS 400 is shown to include BMS controller 366 and a plurality of building subsystems 428. Building subsystems 428 are shown to include a building electrical subsystem 434, an information communication technology (ICT) subsystem 436, a security subsystem 438, a HVAC subsystem 440, a lighting subsystem 442, a lift/escalators subsystem 432, and a fire safety subsystem 430. In various embodiments, building subsystems 428 can include fewer, additional, or alternative subsystems. For example, building subsystems 428 may also or alternatively include a refrigeration subsystem, an advertising or signage subsystem, a cooking subsystem, a vending subsystem, a printer or copy service subsystem, or any other type of building subsystem that uses controllable equipment and/or sensors to monitor or control building 10. In some embodiments, building subsystems 428 include waterside system 200 and/or airside system 300, as described with reference to FIGS. 2-3.

Each of building subsystems 428 may include any number of devices, controllers, and connections for completing its individual functions and control activities. HVAC subsystem 440 may include many of the same components as HVAC system 100, as described with reference to FIGS. 1-3. For example, HVAC subsystem 440 may include a chiller, a boiler, any number of air handling units, economizers, field controllers, supervisory controllers, actuators, temperature sensors, and other devices for controlling the temperature, humidity, airflow, or other variable conditions within building 10. Lighting subsystem 442 may include any number of light fixtures, ballasts, lighting sensors, dimmers, or other devices configured to controllably adjust the amount of light provided to a building space. Security subsystem 438 may include occupancy sensors, video surveillance cameras, digital video recorders, video processing servers, intrusion detection devices, access control devices and servers, or other security-related devices.

Still referring to FIG. 4, BMS controller 366 is shown to include a communications interface 407 and a BMS interface 409. Interface 407 may facilitate communications between BMS controller 366 and external applications (e.g., monitoring and reporting applications 422, enterprise control applications 426, remote systems and applications 444, applications residing on client devices 448, etc.) for allowing user control, monitoring, and adjustment to BMS controller 366 and/or subsystems 428. Interface 407 may also facilitate communications between BMS controller 366 and client devices 448. BMS interface 409 may facilitate communications between BMS controller 366 and building subsystems 428 (e.g., HVAC, lighting security, lifts, power distribution, business, etc.).

Interfaces 407, 409 can be or include wired or wireless communications interfaces (e.g., jacks, antennas, transmitters, receivers, transceivers, wire terminals, etc.) for conducting data communications with building subsystems 428 or other external systems or devices. In various embodiments, communications via interfaces 407, 409 may be direct (e.g., local wired or wireless communications) or via a communications network 446 (e.g., a WAN, the Internet, a cellular network, etc.). For example, interfaces 407, 409 can include an Ethernet card and port for sending and receiving data via an Ethernet-based communications link or network. In another example, interfaces 407, 409 can include a WiFi transceiver for communicating via a wireless communications network. In another example, one or both of interfaces 407, 409 may include cellular or mobile phone communications transceivers. In one embodiment, communications interface 407 is a power line communications interface and BMS interface 409 is an Ethernet interface. In other embodiments, both communications interface 407 and BMS interface 409 are Ethernet interfaces or are the same Ethernet interface.

Still referring to FIG. 4, BMS controller 366 is shown to include a processing circuit 404 including a processor 406 and memory 408. Processing circuit 404 may be communicably connected to BMS interface 409 and/or communications interface 407 such that processing circuit 404 and the various components thereof can send and receive data via interfaces 407, 409. Processor 406 can be implemented as a general purpose processor, an application specific integrated circuit (ASIC), one or more field programmable gate arrays (FPGAs), a group of processing components, or other suitable electronic processing components.

Memory 408 (e.g., memory, memory unit, storage device, etc.) may include one or more devices (e.g., RAM, ROM, Flash memory, hard disk storage, etc.) for storing data and/or computer code for completing or facilitating the various processes, layers and modules described in the present application. Memory 408 may be or include volatile memory or non-volatile memory. Memory 408 may include database components, object code components, script components, or any other type of information structure for supporting the various activities and information structures described in the present application. According to an exemplary embodiment, memory 408 is communicably connected to processor 406 via processing circuit 404 and includes computer code for executing (e.g., by processing circuit 404 and/or processor 406) one or more processes described herein.

In some embodiments, BMS controller 366 is implemented within a single computer (e.g., one server, one housing, etc.). In various other embodiments BMS controller 366 may be distributed across multiple servers or computers (e.g., that can exist in distributed locations). Further, while FIG. 4 shows applications 422 and 426 as existing outside of BMS controller 366, in some embodiments, applications 422 and 426 may be hosted within BMS controller 366 (e.g., within memory 408).

Still referring to FIG. 4, memory 408 is shown to include an enterprise integration layer 410, an automated measurement and validation (AM&V) layer 412, a demand response (DR) layer 414, a fault detection and diagnostics (FDD) layer 416, an integrated control layer 418, and a building subsystem integration later 420. Layers 410-420 may be configured to receive inputs from building subsystems 428 and other data sources, determine optimal control actions for building subsystems 428 based on the inputs, generate control signals based on the optimal control actions, and provide the generated control signals to building subsystems 428. The following paragraphs describe some of the general functions performed by each of layers 410-420 in BMS 400.

Enterprise integration layer 410 may be configured to serve clients or local applications with information and services to support a variety of enterprise-level applications. For example, enterprise control applications 426 may be configured to provide subsystem-spanning control to a graphical user interface (GUI) or to any number of enterprise-level business applications (e.g., accounting systems, user identification systems, etc.). Enterprise control applications 426 may also or alternatively be configured to provide configuration GUIs for configuring BMS controller 366. In yet other embodiments, enterprise control applications 426 can work with layers 410-420 to optimize building performance (e.g., efficiency, energy use, comfort, or safety) based on inputs received at interface 407 and/or BMS interface 409.

Building subsystem integration layer 420 may be configured to manage communications between BMS controller 366 and building subsystems 428. For example, building subsystem integration layer 420 may receive sensor data and input signals from building subsystems 428 and provide output data and control signals to building subsystems 428. Building subsystem integration layer 420 may also be configured to manage communications between building subsystems 428. Building subsystem integration layer 420 translate communications (e.g., sensor data, input signals, output signals, etc.) across a plurality of multi-vendor/multi-protocol systems.

Demand response layer 414 may be configured to optimize resource usage (e.g., electricity use, natural gas use, water use, etc.) and/or the monetary cost of such resource usage in response to satisfy the demand of building 10. The optimization may be based on time-of-use prices, curtailment signals, energy availability, or other data received from utility providers, distributed energy generation systems 424, from energy storage 427 (e.g., hot TES 242, cold TES 244, etc.), or from other sources. Demand response layer 414 may receive inputs from other layers of BMS controller 366 (e.g., building subsystem integration layer 420, integrated control layer 418, etc.). The inputs received from other layers may include environmental or sensor inputs such as temperature, carbon dioxide levels, relative humidity levels, air quality sensor outputs, occupancy sensor outputs, room schedules, and the like. The inputs may also include inputs such as electrical use (e.g., expressed in kWh), thermal load measurements, pricing information, projected pricing, smoothed pricing, curtailment signals from utilities, and the like.

According to an exemplary embodiment, demand response layer 414 includes control logic for responding to the data and signals it receives. These responses can include communicating with the control algorithms in integrated control layer 418, changing control strategies, changing setpoints, or activating/deactivating building equipment or subsystems in a controlled manner. Demand response layer 414 may also include control logic configured to determine when to utilize stored energy. For example, demand response layer 414 may determine to begin using energy from energy storage 427 just prior to the beginning of a peak use hour.

In some embodiments, demand response layer 414 includes a control module configured to actively initiate control actions (e.g., automatically changing setpoints) which minimize energy costs based on one or more inputs representative of or based on demand (e.g., price, a curtailment signal, a demand level, etc.). In some embodiments, demand response layer 414 uses equipment models to determine an optimal set of control actions. The equipment models may include, for example, thermodynamic models describing the inputs, outputs, and/or functions performed by various sets of building equipment. Equipment models may represent collections of building equipment (e.g., subplants, chiller arrays, etc.) or individual devices (e.g., individual chillers, heaters, pumps, etc.).

Demand response layer 414 may further include or draw upon one or more demand response policy definitions (e.g., databases, XML files, etc.). The policy definitions may be edited or adjusted by a user (e.g., via a graphical user interface) so that the control actions initiated in response to demand inputs may be tailored for the user's application, desired comfort level, particular building equipment, or based on other concerns. For example, the demand response policy definitions can specify which equipment may be turned on or off in response to particular demand inputs, how long a system or piece of equipment should be turned off, what setpoints can be changed, what the allowable set point adjustment range is, how long to hold a high demand setpoint before returning to a normally scheduled setpoint, how close to approach capacity limits, which equipment modes to utilize, the energy transfer rates (e.g., the maximum rate, an alarm rate, other rate boundary information, etc.) into and out of energy storage devices (e.g., thermal storage tanks, battery banks, etc.), and when to dispatch on-site generation of energy (e.g., via fuel cells, a motor generator set, etc.).

Integrated control layer 418 may be configured to use the data input or output of building subsystem integration layer 420 and/or demand response later 414 to make control decisions. Due to the subsystem integration provided by building subsystem integration layer 420, integrated control layer 418 can integrate control activities of the subsystems 428 such that the subsystems 428 behave as a single integrated super-system. In an exemplary embodiment, integrated control layer 418 includes control logic that uses inputs and outputs from a plurality of building subsystems to provide greater comfort and energy savings relative to the comfort and energy savings that separate subsystems could provide alone. For example, integrated control layer 418 may be configured to use an input from a first subsystem to make an energy-saving control decision for a second subsystem. Results of these decisions can be communicated back to building subsystem integration layer 420.

Integrated control layer 418 is shown to be logically below demand response layer 414. Integrated control layer 418 may be configured to enhance the effectiveness of demand response layer 414 by enabling building subsystems 428 and their respective control loops to be controlled in coordination with demand response layer 414. This configuration may advantageously reduce disruptive demand response behavior relative to conventional systems. For example, integrated control layer 418 may be configured to assure that a demand response-driven upward adjustment to the setpoint for chilled water temperature (or another component that directly or indirectly affects temperature) does not result in an increase in fan energy (or other energy used to cool a space) that would result in greater total building energy use than was saved at the chiller.

Integrated control layer 418 may be configured to provide feedback to demand response layer 414 so that demand response layer 414 checks that constraints (e.g., temperature, lighting levels, etc.) are properly maintained even while demanded load shedding is in progress. The constraints may also include setpoint or sensed boundaries relating to safety, equipment operating limits and performance, comfort, fire codes, electrical codes, energy codes, and the like. Integrated control layer 418 is also logically below fault detection and diagnostics layer 416 and automated measurement and validation layer 412. Integrated control layer 418 may be configured to provide calculated inputs (e.g., aggregations) to these higher levels based on outputs from more than one building subsystem.

Automated measurement and validation (AM&V) layer 412 may be configured to verify that control strategies commanded by integrated control layer 418 or demand response layer 414 are working properly (e.g., using data aggregated by AM&V layer 412, integrated control layer 418, building subsystem integration layer 420, FDD layer 416, or otherwise). The calculations made by AM&V layer 412 may be based on building system energy models and/or equipment models for individual BMS devices or subsystems. For example, AM&V layer 412 may compare a model-predicted output with an actual output from building subsystems 428 to determine an accuracy of the model.

Fault detection and diagnostics (FDD) layer 416 may be configured to provide on-going fault detection for building subsystems 428, building subsystem devices (i.e., building equipment), and control algorithms used by demand response layer 414 and integrated control layer 418. FDD layer 416 may receive data inputs from integrated control layer 418, directly from one or more building subsystems or devices, or from another data source. FDD layer 416 may automatically diagnose and respond to detected faults. The responses to detected or diagnosed faults may include providing an alert message to a user, a maintenance scheduling system, or a control algorithm configured to attempt to repair the fault or to work-around the fault.

FDD layer 416 may be configured to output a specific identification of the faulty component or cause of the fault (e.g., loose damper linkage) using detailed subsystem inputs available at building subsystem integration layer 420. In other exemplary embodiments, FDD layer 416 is configured to provide “fault” events to integrated control layer 418 which executes control strategies and policies in response to the received fault events. According to an exemplary embodiment, FDD layer 416 (or a policy executed by an integrated control engine or business rules engine) may shut-down systems or direct control activities around faulty devices or systems to reduce energy waste, extend equipment life, or assure proper control response.

FDD layer 416 may be configured to store or access a variety of different system data stores (or data points for live data). FDD layer 416 may use some content of the data stores to identify faults at the equipment level (e.g., specific chiller, specific AHU, specific terminal unit, etc.) and other content to identify faults at component or subsystem levels. For example, building subsystems 428 may generate temporal (i.e., time-series) data indicating the performance of BMS 400 and the various components thereof. The data generated by building subsystems 428 may include measured or calculated values that exhibit statistical characteristics and provide information about how the corresponding system or process (e.g., a temperature control process, a flow control process, etc.) is performing in terms of error from its setpoint. These processes can be examined by FDD layer 416 to expose when the system begins to degrade in performance and alert a user to repair the fault before it becomes more severe.

BMS Performance Index (BPI)

In some embodiments, a performance index is calculated for a BMS (e.g., the BMS of building 10, described above). This BMS performance index (BPI) may be a value indicative of the overall health and efficiency of a BMS. A BPI value can provide numerous insights to a user (e.g., a building manager, a facilities operator, etc.), allowing the user to quickly and easily determine whether the BMS is running optimally. Advantageously, BPI can be calculated for one or more BMSs managed by a single user, system, group, company, etc., providing an overview of BMS health and efficiency across multiple sites, buildings, facilities, etc. Additionally, a BPI for a first site can be compared to other sites having similar parameters (e.g., location, size, building type, etc.) to provide insights regarding the first site's efficiency compared to other sites.

BPI may also provide a user with information regarding various BMS inefficiencies. For example, an “ideal” BMS would operate almost completely autonomously, requiring very little user input, because many BMSs are designed to optimize energy efficiency, reduce operating costs, increase equipment longevity, etc. Many things can negatively impact BMS performance, however. For example, out-of-date software and firmware may be slower, more inefficient, and/or more susceptible to cyberattacks than current software/firm. Additionally, equipment or setpoints that are being manually controlled by a user (e.g., by overriding a setpoint or placing the equipment in a manual operating mode) cannot be optimized by the BMS, leading to decreased system efficiency.

Accordingly, via the process of calculating a BPI for a BMS, these BMS inefficiencies or issues can be identified, allowing the user to correct certain deficiencies to improve BMS performance. For example, a user may not be aware that certain building devices were placed into a manual operating mode (e.g., for service), which may be revealed via the BPI calculation, allowing the user to place the equipment back into an automatic mode. Similarly, it may be difficult for a user to track hundreds of setpoints, so it may not be uncommon for overridden setpoints to forgotten, negatively impacting BMS performance. In some cases, these overridden setpoints may be identified, allowing the user to quickly revert the setpoint back to an automatic or recommended value.

Referring now to FIG. 5, a block diagram of a system 500 for monitoring various components of a BMS and calculating a BPI value is shown, according to some embodiments. System 500 is shown to include a BPI tool 600, a site analytics tool 502, and a gateway 504, all communicably coupled to network 446. While shown as singular components each of BPI tool 600, site analytics tool 502, and gateway 504 may also be implemented across multiple devices (e.g., via a distributed computing architecture).

BPI tool 600, site analytics tool 502, and gateway 504 generally include a processor and memory for storing and executing instructions. Said memory may include one or more devices (e.g., RAM, ROM, Flash memory, hard disk storage, etc.) for storing data and/or computer code for completing or facilitating the various processes, layers and modules described in the present application. The memory may be or include volatile memory or non-volatile memory and may include database components, object code components, script components, or any other type of information structure for supporting the various activities and information structures described in the present application.

At a high level, BPI tool 600, site analytics tool 502, and gateway 504 may transmit and receive (i.e., communicate) data, including various operating data and/or parameters, via network 446. For example, BPI tool 600 may receive data from site analytics tool 502 and gateway 504, and/or may transmit data to site analytics tool 502 and gateway 504. Network 446 may be any type of communications network, as described above, such as a WAN, LAN, the Internet, a cellular network, etc. Accordingly, each of BPI tool 600, site analytics tool 502, and gateway 504 may include a network interface for wired and/or wireless communications. For example, BPI tool 600 may include a wireless network interface (e.g., a WiFi transmitter/receiver) and site analytics tool may include an Ethernet interface. It will be appreciated that any combination of wired and/or wireless communications may be utilized.

As shown, gateway 504 can be configured to receive operating data from a plurality of supervisory controllers 510-514. In particular, in some embodiments, gateway 504 may receive and/or collect operating data via an open communications protocol, such as BACnet, from any of supervisory controllers 510-514. Supervisory controllers 510-514 may be any high-level controller of a BMS, such as network automation engines (NAEs), network integration engines (NIEs), or network control engines (NCEs). In one example, each of building subsystems 426, described above, may include a supervisory controller. In some embodiments, each of supervisory controllers 510-514 can include a processor and memory for performing one or more functions, such as receiving, processing, and/or transmitting data, and/or providing control signals to various lower-level field controllers 516-526. Additionally, it will be appreciated that system 500 may include any number of supervisory controllers.

Field controllers 516-526 can include any controllers in a BMS that are at a lower level (e.g., hierarchically) than supervisory controllers 510-514. For example, each of field controllers 516-526 may be a controller for a particular device or space in a building. Supervisory controllers 510-514 may receive operating data from field controllers 516-526 relating to various sensors and/or devices 528 monitored and/or controlled by field controllers 516-526. Sensors/device 528 can include any sensors or field devices (i.e., edge devices) included in a BMS, such as any of the sensors or equipment described above with respect to FIGS. 1-4. As an example, sensors/devices 528 can include sensors for measuring various equipment or space parameters (e.g., temperature, pressure, speed, etc.) and devices such as chillers, AHUs, valves, lights, fans, etc.

In some embodiments, field controllers 516-526 collect operating data from sensors/devices 528 during BMS operations, and also provide control signals to sensors/devices 528 based on the operating data, and/or based on other inputs. For example, a field controller for a chiller may collect information such as pump speed, input/output pressure and temperature, etc., and may also provide control signals that cause the chiller to maintain a certain setpoint (e.g., outlet water temperature). Supervisory controllers 510-514 can subsequently collect said operating data, and other information such a field controller parameters or settings, from one or more of field controllers 516-526. It will be appreciated that, as described herein, operating data may be collected and/or transmitted on demand, at regular intervals, instantly, or at any other appropriate interval.

To continue the previous example, a field controller (e.g., field controller 516) for a chiller can collect various operating data in real-time, as the chiller operates. This operating data may then be collected, in real-time or at a regular interval (e.g., every five minutes, every hour, etc.), by a corresponding supervisory controller (e.g., supervisory controller 510, which may be a supervisory controller for an HVAC subsystem such as HVAC subsystem 440). A portion of the operating data may be formatted in accordance with an open communications protocol (e.g., BACnet), as discussed above, and accordingly may be collected and transmitted (e.g., to BPI tool 600 and/or site analytics tools 502) by gateway 504.

In some embodiments, a portion of the operating data collected by supervisory controllers 510-514 may be in a proprietary format that cannot be received, processed, and/or transmitted by gateway 504. In other words, certain operating data may be collected from proprietary equipment or sensors (e.g., sensors/devices 528) in a format other that an open communication protocol. Additionally, information such as parameters and/or settings (e.g., setpoints, schedules, etc.) of field controllers 516-526 and/or supervisory controllers 510-514 may not be accessible or receivable by gateway 504. In this case, an application data server (ADS) 506 may collect a portion of the operating data for further analysis and/or processing, before being transmitted to BPI tool 600 and/or site analytics tool 502.

ADS 506 may be a computing device such as a server or computer that manages the collection of large amounts of operating data from the various components of a BMS. In this case, ADS 506 is configured to collect operating data and other information from supervisory controllers 510-514. In particular, ADS 506 can collect data in both an open communication protocol, and any other formats (e.g., a proprietary format). Accordingly, ADS 506 can process and/or reformat the data that cannot be handled by gateway 504. ADS 506 may also or host a performance verification tool (PVT) 508, which processes and/or reformats the collected operating data. In particular, PVT 508 can obtain and analyze the operating data, and can generate a report or can convert the operating data for transmission to BPI tool 600 and/or site analytics tool 502 by gateway 504. In some embodiments, PVT 508 is an application or a program that is stored on memory of ADS 506 and executed by a processor of ADS 506.

In some embodiments, PVT 508 is continuously executed, thereby processing the operating data in real-time. In other embodiments, PVT 508 is executed at a regular interval (e.g., every day) to batch process the operating data. In such embodiments, ADS 506 may collect the operating data between executions of the PVT 508, and PVT 508 may generate a report based on the collected data. This report and/or the processed operating data may be transmitted via gateway 504 to BPI tool 600 and/or site analytics tool 502.

Site analytics tool 502 is generally configured to receive raw or preprocessed operating data from gateway 504 (e.g., via an application programming interface (API), in some cases), and can perform various additional functions using the data. In particular, site analytics tool 502 may be configured to aggregate portions of the operating data, and may also identify faults, warnings, or alarms. For example, site analytics tool 502 may analyze operating data to determine a fault with a particular building device (e.g., of sensors/devices 528), and may provide an alarm or notification based on the fault. In some embodiments, site analytics tool 502 may also generate user interfaces for presenting aggregate operating data in the form of graphs, charts, etc., and for presenting fault or alarm information. In some such embodiments, site analytics tool 502 may implement various FDD rules (e.g., similar to FDD layer 416 of BMS controller 366), and/or may interface with BMS controller 366 to identify faults.

Referring now to FIG. 6, a detailed block diagram of BPI tool 600 is shown, according to some embodiments. As briefly described above, BPI tool 600 may receive data from site analytics tool and/or gateway 504, and may calculate a BPI based on the received data. In particular, BPI tool 600 may receive operating data or parameters relating to ADS 506, supervisory controllers 510-514, field controllers 516-526, and/or sensors/devices 528, and may generate a performance score for each components of system 500. Performance scores for each of the components of system 500 may be aggregated to calculate a BPI. Based on the BPI, which can be calculated once (e.g., on demand) or periodically, or even continuously, various control actions can be initiated, recommendations can be made, and/or interventions can be initiated.

BPI tool 600 is shown to include a processing circuit 602, which includes a processor 604 and a memory 610. It will be appreciated that these components can be implemented using a variety of different types and quantities of processors and memory. For example, processor 604 can be a general purpose processor, an application specific integrated circuit (ASIC), one or more field programmable gate arrays (FPGAs), a group of processing components, or other suitable electronic processing components. Processor 604 can be communicatively coupled to memory 610. While processing circuit 602 is shown as including one processor 604 and one memory 610, it should be understood that, as discussed herein, a processing circuit and/or memory may be implemented using multiple processors and/or memories in various embodiments. All such implementations are contemplated within the scope of the present disclosure.

Memory 610 can include one or more devices (e.g., memory units, memory devices, storage devices, etc.) for storing data and/or computer code for completing and/or facilitating the various processes described in the present disclosure. Memory 610 can include random access memory (RAM), read-only memory (ROM), hard drive storage, temporary storage, non-volatile memory, flash memory, optical memory, or any other suitable memory for storing software objects and/or computer instructions. Memory 610 can include database components, object code components, script components, or any other type of information structure for supporting the various activities and information structures described in the present disclosure. Memory 610 can be communicably connected to processor 604 via processing circuit 602 and can include computer code for executing (e.g., by processor 604) one or more processes described herein.

Memory 610 is shown to include a data analyzer 612, configured to processes a wide variety of operating data from site analytics tools 502 and/or gateway 504. In particular, BPI tool 600 may receive operating data from site analytics tools 502 and/or gateway 504, and data analyzer may interpret, format, store, and/or retrieve the operating data. In some embodiments, data analyzer 612 requests (i.e., queries) particular operating data needed for calculating a BPI, as discussed in greater detail below. In such embodiments, data analyzer 612 may transmit queries to any of site analytics tools 502, gateway 504, or ADS 506 (e.g., through gateway 504), and may subsequently receive requested information. In some embodiments, data analyzer 612 receives raw operating data from gateway 504 and receives pre-processed data from site analytics tool 502. In particular, site analytics tool 502 may transmit fault and alarm data to data analyzer 612.

Memory 610 is also shown to include a performance index generator 614, configured to generate a BPI value based on operating data. Performance index generator 614 may receive any operating data necessary for calculating the BPI from data analyzer 612, and may calculate the BPI on demand, or at regular intervals. In some embodiments, performance index generator 614 calculates a performance score for each component or for various subsets of the components of system 500. In particular, performance index generator 614 may generate individual performance scores for ADS 506, supervisory controllers 510-514, field controllers 516-526, and sensors/devices 528. In some cases, performance index generator 614 even generates a performance score for various subsets of sensors/devices 528, such as for particular types of devices (e.g., valves, chillers, etc.) or sensors (e.g., types of sensors, spaces or equipment associated with the sensors, etc.). Performance index generator 614 may aggregate the various individual performance scores to generate the BPI.

In some embodiments, BPI tool 600 receives operating data in real-time, or in near real-time. Accordingly, a BPI may be calculated continuously, or at regular intervals. In some embodiments, BPI tool 600 may analyze previously collected (i.e., historic) BMS data to calculate a BPI at a previous time period. Accordingly, BPI can be compared over time, to determine if certain systems changes or upgrades (e.g., new devices) are beneficial to system health and efficiency.

In some embodiments, performance index generator 614 may analyze the operating data according to a plurality of rules. Each rule may define a calculation or model for determining a penalty score for a particular parameter of an associated BMS component, and penalty scores for each rule may be applied to the performance score for each component. When analyzing a supervisory controller or multiple controllers, for example, performance index generator 614 may utilize a variety of predefined parameters (e.g., firmware version, memory usage, operating temperature, etc.) that affect the health and/or efficiency of the supervisory controller(s). If it is determined that a particular controller or controllers do not meet a parameter (e.g., memory usage too high, operating temperature too high, etc.), then performance index generator 614 will apply a penalty to the performance score for the controller(s).

The calculations performed by performance index generator 614 are described in greater detail with respect to FIG. 7, which shows an example graph 700 for calculating the BPI of a BMS, according to some embodiments. Based on the calculations shown in graph 700, performance index generator 614 may calculate individual performance scores for one or more ADSs, supervisory controllers, field controllers, sensors, actuating device, points, and various other equipment of a BMS. In this example, the BMS includes one ADS (e.g., ADS 506), twenty supervisory controllers (e.g., supervisory controllers 510-514), and ten field controllers (e.g., field controllers 516-526). For each parameter or rule, graph 700 includes a brief description, a total count of components (e.g., devices, sensors, points, etc.), a total deviation count (e.g., the number of components that do not meet the rule), an ideal score, and a penalty score.

As shown, performance index generator 614 may obtain (e.g., automatically or by request) a variety of operating data for each component, based on the various parameters or rules that are analyzed to generate the performance score for each component. To analyze ADS servers, for example, performance index generator 614 may determine a version of the ADS firmware or software, average CPU and RAM usages over the ADS, and an amount of free disk space on the ADS. As discussed above, this operating data may be received by BPI tool 600 from ADS 506 (e.g., via gateway 504). For example, ADS 506 may regularly transmit this operating data to BPI tool 600, or BPI tool 600 may request or determine this information as needed. In some embodiments, BPI tool 600 may interface with an API or an application hosted by ADS 506 that collects this information.

Continuing the example shown in FIG. 7, the single ADS for this BMS (e.g., the BMS of building 10) is determined to meet the parameters for CPU usage, RAM usage, and free disk space, but fails to meet the parameter for ADS version. In other words, the average CPU usage and RAM usage for this ADS are below 75%, and more than 10% disk space is available; however, the ADS version is not up to date. Accordingly, a total deviation of ‘1’ is recorded in the “total deviation count” column of graph 700, while zeros are recorded for the remaining ADS parameters. A penalty score is then calculated for each parameter. Taking the “ADS version” parameter as an example, the penalty score is equal to the ideal score if the ADS version is determined to be older than the current (i.e., up-to-date) version. In this case, because the ADS is out of date and the ideal score for ADS version is four, a penalty score of four is calculated for this parameter. However, because the other ADS-specific parameters (e.g., CPU and RAM usage, free disk space) are below the required thresholds (e.g., 75% and 10%, respectively), the penalty for these parameters is zero. Hence, the overall performance score for the ADS is 6.0, which is lower than the greatest actual score of 10.

In another example based on graph 700, the example BMS includes 150 sensors. Based on operating data received from gateway 504 and/or site analytics tools 502, BPI tool 600 has determined that there are current seven sensors associated with some sort of sensor alarm, and three sensors associated with a sensor fault. A sensor alarm may indicate that a sensor value has exceed a threshold (e.g., a temperature exceeding an upper limit, as determined by a temperature sensor), for example, while a sensor fault may indicate that the sensor is not working correctly (e.g., not transmitting data, transmitting erroneous data, etc.). A penalty score is calculated for the number of sensor alarms and faults, where the penalty score is equal to:

$S_{penalty} = {\left( \frac{s_{ideal}}{n} \right) \times z}$

where S_(penalty) is the penalty score, S_(ideal) is the predetermined ideal score, n is the total number of sensors, and z is the number of sensors in fault or alarm states, respectively. Here, a performance score of 9.7 is calculated for the sensors, which is slightly lower than the greatest actual score of 10.

After calculating a performance score for each component of the BMS, performance index generator 614 may aggregate the scores to determine the BPI of the BMS. In this example, the BPI is calculated at 88.20, out of a total possible BPI of 100 (e.g., where a BPI of 100 would be an ideal system). In some embodiments, performance index generator 614 may also identify (e.g., flag) parameters or rules that the BMS did not meet. In other words, performance index generator 614 may indicate areas where the BMS was issued a penalty score. For example, in response to determining that three field controllers are offline (e.g., as shown in graph 700), performance index generator 614 may identify the offline controllers for additional analysis or manual inspection by a user. The process of calculating a BPI for a BMS is described below in greater detail, with respect to FIG. 8.

Referring again to FIG. 6, memory 610 further includes a user interface (UI) generator 616. UI generator 616 may be configured to generate graphical interfaces for presenting BPI related information. For example, UI generator 616 may generate interfaces that present graphs, charts, animations, etc. that indicate a BPI value for a BMS. In some embodiments, UI generator 616 may also generate and present interfaces that indicate BMS inefficiencies. For example, an interface may be generated that indicates specific BMS components (e.g., controllers, devices, sensors, etc.) that caused a non-zero penalty score during the calculation of the BPI. The various user interfaces generated by UI generator 616 are described in greater detail below, with respect to FIGS. 9 and 10.

The various user interfaces generated by UI generator 616 may be presented via a user device 632. User device 632 may be any device having an interface for presenting data to a user. For example, user device 632 may include at least a screen for presenting interfaces, and an input device for receiving user inputs. In some embodiments, user device 632 is a desktop or laptop computer, a smartphone, a tablet, a smart watch, etc. User device 632 may be communicably coupled to BPI tool 600 via a communications interface 630, which also provides an interface for BPI tool 600 to transmit and receive data via network 446.

Memory 610 also includes a database 618, which can be configured to store, maintain, and/or retrieve any type of information that is relevant to the calculation of a BPI. For example, database 618 may store operating data received from any of the components of system 500, and/or may store previous BPI calculations. In this regard, the BPI for a particular BMS may be analyzed (e.g., via a user interface) over time, to identify trends that indicate increased or decreased system health and efficiency. For example, a BPI that steadily rises over time may indicate that various operating processes are improving system efficiency.

Referring now to FIG. 8, a process 800 for calculating a BPI value for a BMS is shown, according to some embodiments. Process 800 may be implemented by system 500, for example, and at least in part by BPI tool 600. As described above, BPI may be calculated for one or more facilities (i.e., sites, buildings, etc.), and may provide an intuitive indication of the health and/or efficiencies of the facilities. In particular, BPI can be a numerical value that indicates BMS performance. Advantageously, process 800 may allow users (e.g., building managers) to quickly identify BMS inefficiencies and trends, leading to improved BMS operations. It will be appreciated that certain steps of process 800 may be optional and, in some embodiments, process 800 may be implemented using less than all of the steps.

At step 802, first operating data for one or more servers (e.g., computing devices, computers) of a BMS is obtained. In some embodiments, the one or more servers include at least a main computing device for a BMS, such as a BMS controller or a device that executes BMS software. In other words, the one or more servers can include any high-level computing devices of a BMS. In system 500, for example, the first data includes operating parameters of ADS 506. More specifically, the first data includes operating parameters associated with one or more rules of a BPI model (e.g., graph 700). In some embodiments, the first data includes at least an indication of a software or firmware version, an average CPU and RAM usage, and an amount of free disk space for each of the one or more high-level computing devices.

At step 804, a first performance score is calculated based on the first operating data. The first performance score may indicate the performance (e.g., health and efficiency) of the one or more servers. In system 500, for example, the first performance score may indicate a health and efficiency of ADS 506. The first performance score may be calculated by first determining a one or more rules or parameters for the servers. These rules may be defined in a BPI model (e.g., performance index generator 614), for example.

As discussed above with respect to FIG. 7, the parameters for servers can include a minimum software/firmware version, a maximum average CPU usage, a maximum average RAM usage, and a minimum free disk space. The first data obtained at step 802 may be utilized to determine whether the servers fail to meet one or more parameters. Any parameters that the servers fail to meet may have a penalty score applied. For example, a server device (e.g., ADS 506) that exceeds the maximum average CPU usage for a designated amount of time (e.g., two minutes) may have a penalty score applied for that parameter.

Penalty scores may be subtracted from an ideal score for each parameter. Using graph 700 as an example, an ADS server with an out-of-date software version may have a penalty score equal to the ideal score applied. Subtracting the penalty score from the ideal score would result in a 0 actual score for that parameter. The actual scores from each parameter associated with the servers may then be aggregated to determine an actual overall score (i.e., a first performance score) for the components (e.g., the servers).

At step 806, second operating data for one or more controllers of a BMS is obtained. Specifically, the second operating data may include operating data for one or more supervisory controllers and/or one or more field controllers of the BMS. In some embodiments, the second data may be collected in part by the servers (e.g., via a program such as the performance verification tool 508), and may also be collected in part by gateway 504. For example, operating data in a first, open format (e.g., BACnet) may be collected by gateway 504, while data in a second, proprietary format may be collected by performance verification tool 508. In some embodiments, the second data includes at least an indication of network status (e.g., online or offline) for each controller, and an indication of a firmware version for each controller. Additionally, for supervisory controllers, the second data may indicate an average memory usage, a temperature, and a battery status for each controller.

At step 808, a second performance score is calculated based on the second operating data. Once again, the second performance score may indicate the performance (e.g., health and efficiency) of the one or more supervisory and/or field controllers within a BMS. In system 500, for example, the second performance score may indicate a health and efficiency of supervisory controllers 510-514 and/or field controllers 516-526. The second performance score may be calculated by first determining a one or more rules or parameters for the controllers. These parameters, for example, may define at least a current firmware version and a desired network status for the controllers.

In some embodiments, the parameters for supervisory controllers may also include a maximum average memory usage, an operating temperature range, and a desired battery level. In some such embodiments, battery level may simply be determined by an indication that the battery is low (e.g., below a threshold capacity). The second operating data obtained at step 806 may be utilized to determine whether one or more controllers fail to meet any of the one or more parameters. As discussed above with respect to the servers, a penalty score may be applied for any controllers that do not meet a parameter.

As an example, an “ideal” supervisory controller may be online and have up-to-date firmware, a memory usage below a threshold percentage (e.g., 80%), an operating temperature between a minimum and maximum value (e.g., 20° C.<T<40° C.), and a full charged battery. In this example, any supervisory controller that doesn't meet one or more parameters may have a penalty score applied. Penalty scores may be determined by a formula unique to each parameter or rule. For example, the penalty score for average memory usage may be determined by:

$S_{penalty} = {\left( \frac{s_{ideal}}{n} \right) \times z}$

where S_(penalty) is the penalty score, S_(ideal) is the predetermined ideal score, n is the total number of supervisory controllers, and z is the number of controllers have a memory usage greater than a threshold percentage. Additional penalty score calculations are shown in graph 700, described above.

In any case, the penalty score for each parameter may be subtracted from the ideal score for each parameter to determine an actual parameter score. For example, a BMS with three offline field controllers, out of ten total field controllers, may have a penalty score of 4.5. If an ideal score for the network status parameter of the field controllers is 15, then the actual score for that parameter will be 10.5. The actual scores for each parameter may then be aggregated to determine an actual overall score for the component(s) (i.e., a second performance score). In graph 700, for example, the actual overall score for field controllers was 15.5 out of a maximum possible score of 20.

At step 810, third operating data for one or more edge devices of a BMS is obtained. In particular, the third operating data may include operating data for one or more sensors and actuating devices within the BMS; however, in some embodiments, the third data also includes operating data for other equipment (e.g., any building devices shown in FIGS. 1-4). In some embodiments, the third operating data includes an indication of sensor alarm or faults for any sensor devices included in a BMS. In some embodiments, the third operating data includes an indication of whether one or more actuating devices are in a manual operating state, and whether the one or more actuating devices are stuck (e.g., open or closed). A stuck actuating device may be associated with a fault, for example.

In some embodiments, the third operating data may also indicate whether one or more setpoints within a BMS (e.g., temperature, pressure, etc.) have been overridden. For example, a temperature setpoint for a room that has been manually set to 75° F. may be indicated as an “overridden” setpoint. The third operating data may also indicate whether one or more building devices (e.g., other than sensors and actuating devices) are operating in a manual mode, as opposed to an automatic mode. In some embodiments, the third operating data is collected by field controllers 516-526 and subsequently transmitted to supervisory controllers 510-514, and then to BPI tool 600 via gateway 504.

At step 812, a third performance score is calculated based on the third operating data. The third performance score may indicate the performance (e.g., health and efficiency) of the building devices (e.g., equipment, sensors, etc.) within a BMS. In system 500, for example, the second performance score may indicate a health and efficiency of sensors/devices 528. The third performance score may also indicate overall system performance, where a system operating almost completely automatically (e.g., within devices in manual operating modes and/or overridden setpoints) may be much more efficient than a system having many devices in manual modes and/or overridden setpoints. As described above with respect to steps 804 and 808, the third performance score may be calculated by first determining a one or more rules or parameters for the building devices.

In some embodiments, the parameters for sensors may simply require a determination that each sensor is associated with an alarm or fault. For example, a site analytics tools (e.g., site analytics tools 502) may monitor various sensors and, if a sensor values exceeds a threshold, may initiate an alarm. In another example, a sensor that provides erratic values may be determined to be faulty, and may be flagged as such. In any case, a penalty may be applied for each sensor that is associated with an alarm or fault. In graph 700, for example, seven out of 150 sensor are associated with faults, resulting in a penalty of 0.2. This penalty is subtracted from the ideal score of five, to result in an actual score of 4.8.

In some embodiments, a penalty score may also be applied for any actuating devices that are in a manual operating mode, or that are indicated as “stuck” (i.e., are experiencing a fault). Using graph 700 again as an example, eight actuating devices are in a manual operating mode, resulting in a penalty score of 0.2. Subtracting the penalty score from the ideal score of two, leading to an actual score of 1.8. Similarly, penalty scores may be applied for any setpoints that are overridden (e.g., 11 out of 250 setpoints, equaling a penalty of 0.4) and for any other devices or equipment that are in a manual operating mode. The actual scores for each parameter may be aggregated to determine, for each type of component (e.g., for all sensors or all actuating devices), an actual overall score (i.e., a third performance score).

At step 814, the first, second, and third scores are aggregated to generate a performance index (e.g., BPI) for the BMS. More specifically, the overall actual scores for each component type or category may be aggregated to determine the BPI for the BMS. In some embodiments, the BPI may include an aggregate of the performance scores for servers, supervisory controllers, field controllers, sensors, actuating devices, setpoints, and other building devices. In graph 700, for example, the actual overall scores for each category are added to determine a BPI of 88.20, out of a maximum possible BPI of 100. In some embodiments, BPI may also be represented as a percentage (e.g., 88.2%) of a maximum value, where the closer the BPI is to a maximum (e.g., 100%), the healthier the BMS.

At step 816, various actions are initiated based on the performance index (i.e., BPI). In some embodiments, these actions include generating recommendations for improving the BPI, and thereby improving BMS performance. A recommendation may include, for example, an indication of one or more parameters or BMS components (e.g., controllers, sensors, etc.) that are negatively impacting the BPI (e.g., parameters with a high penalty score), and may also include an indication of how the BPI may be raised. For example, a high penalty score due to offline field controllers may be lowered (e.g., thereby improving the BPI) by ensuring the field controllers are online (e.g., by manually connecting the controllers to a network, by resetting the controllers, etc.). In this example, a prompt may be provided to a user to manually inspect the offline controllers, and to return them to an online status.

In some embodiments, maintenance or service may be automatically scheduled based on the BPI. In other words, any components that are negatively impacting the BPI may be identified, and some maintenance or service action may be scheduled to correct issues. For example, if a stuck actuating device is lowering the BPI (e.g., by incurring a penalty score), maintenance may be scheduled to fix and/or replace the actuating device. In some embodiments, maintenance is scheduled by transmitting a request (e.g., from BPI tool 600) to a remote maintenance management system.

In some embodiment, one or more building devices may be controlled based on the BPI (e.g., to improve the BPI). For example, if a software version for an upper-level computing device is out-of-date, causing a lower-than-ideal BPI, a remote system may be automatically queried for a new software file, and the updated software file may be automatically installed. In some embodiments, the automated control actions include generating and transmitting a notification (e.g., a push notification, a text message, an email, etc.) to a user's computing device. For example, the calculated BPI may be automatically displayed in a user interface on the user's device, along with an indication of the components or parameters that are negatively impacting the BPI.

In some embodiments the generation and display of a user interface that displays BPI information may be initiated, in response to the calculation of the BPI. For example, the BPI may be displayed via multiple graphical components (e.g., charts, graphs, etc.), such as those shown in FIGS. 9 and 10. Additionally, the user interfaces may display information for improving the BPI, such as by indicating devices or components that are associated with penalty scores. For example, the user interfaces may present recommendations, as discussed above, that are generated based on the BPI. In this regard, the user interfaces may allow a user to quickly determine an overall system health and efficiency, as well as to quickly identify areas of improvement.

Referring now to FIGS. 9 and 10, example interfaces for graphically presenting the BPI of one or more BMSs are shown, according to some embodiments. In particular, FIG. 9 shows an example interface 900 for presenting BPI anomalies across multiple facilities (i.e., sites, buildings, etc.). Interface 900 includes a plurality of graphical elements, in this case graphically distinct circles 902, 903, 904, 905, 906, 907, and 908, which represent a range of BPI values for each of seven different facilities. The size of each graphical element (e.g., circle) may indicate how greatly the BPI for a corresponding facility has varied over time. For example, the BPI of circle 906 corresponding to “Facility 5” has stayed relatively steady near 100 over a one month period, while the BPI of circle 907 corresponding with “Facility 6” has varied greatly over the month (e.g., from approximately 70 to 90). In this regard, a smaller circle may indicate a more stable and/or efficient BMS, while a larger circle may indicate instability. Interface 900 is also shown to include a graphical element 910 for selecting a time period to view BPI anomalies. For example, the user can view BPI anomalies over a one-week period, one-month, three-months, six-months, etc.

FIG. 10 shows an example interface 1000, presenting BPI scores 1001 and penalties 1003 for a particular BMS, over time. Like interface 900, a user can select a time period (e.g., 1 week, 1-month, 6-months, etc.) over which to view BPI values at graphical element 1010. In this case, a range of BPIs are shown over a 10-day period. In some embodiments, the BPI for this particular BMS may have been calculated once each day. For example, on April 28^(th), the BPI is shown in a “medium BPI” zone 1004 on the chart, indicating that the BPI is lower than an upper threshold (e.g., a “high BPI” zone 1002), but higher than a lower threshold (e.g., a “low BPI” zone 1006). The “medium BPI” zone 1004 may indicate that the BPI could be improved, but that the BMS is not overly inefficient. The “low BPI” zone 1006, on the other hand, may indicate that the BMS needs more immediate attention. For each day, the total penalty score is also provided in a bar graph. For example, on April 28^(th), a penalty score of roughly 25 was applied during the BPI calculation.

Configuration of Exemplary Embodiments

The construction and arrangement of the systems and methods as shown in the various exemplary embodiments are illustrative only. Although only a few embodiments have been described in detail in this disclosure, many modifications are possible (e.g., variations in sizes, dimensions, structures, shapes and proportions of the various elements, values of parameters, mounting arrangements, use of materials, colors, orientations, etc.). For example, the position of elements may be reversed or otherwise varied and the nature or number of discrete elements or positions may be altered or varied. Accordingly, all such modifications are intended to be included within the scope of the present disclosure. The order or sequence of any process or method steps may be varied or re-sequenced according to alternative embodiments. Other substitutions, modifications, changes, and omissions may be made in the design, operating conditions and arrangement of the exemplary embodiments without departing from the scope of the present disclosure.

The present disclosure contemplates methods, systems and program products on any machine-readable media for accomplishing various operations. The embodiments of the present disclosure may be implemented using existing computer processors, or by a special purpose computer processor for an appropriate system, incorporated for this or another purpose, or by a hardwired system. Embodiments within the scope of the present disclosure include program products including machine-readable media for carrying or having machine-executable instructions or data structures stored thereon. Such machine-readable media can be any available media that can be accessed by a general purpose or special purpose computer or other machine with a processor. By way of example, such machine-readable media can comprise RAM, ROM, EPROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to carry or store desired program code in the form of machine-executable instructions or data structures and which can be accessed by a general purpose or special purpose computer or other machine with a processor. When information is transferred or provided over a network or another communications connection (either hardwired, wireless, or a combination of hardwired or wireless) to a machine, the machine properly views the connection as a machine-readable medium. Thus, any such connection is properly termed a machine-readable medium. Combinations of the above are also included within the scope of machine-readable media. Machine-executable instructions include, for example, instructions and data which cause a general purpose computer, special purpose computer, or special purpose processing machines to perform a certain function or group of functions.

Although the figures show a specific order of method steps, the order of the steps may differ from what is depicted. Also two or more steps may be performed concurrently or with partial concurrence. Such variation will depend on the software and hardware systems chosen and on designer choice. All such variations are within the scope of the disclosure. Likewise, software implementations could be accomplished with standard programming techniques with rule based logic and other logic to accomplish the various connection steps, processing steps, comparison steps and decision steps. 

What is claimed is:
 1. A system comprising: one or more memory devices having instructions stored thereon that, when executed by one or more processors, cause the one or more processors to perform operations comprising: obtaining first data indicating operating parameters of a server of a building management system (BMS); obtaining second data regarding indicating operating parameters of at least one of a supervisory controller or a field controller of the BMS; obtaining third data regarding indicating operating parameters of one or more edge building devices; and calculating a performance index value for the BMS based on an aggregate of the first data, the second data, and the third data.
 2. The system of claim 1, the operations further comprising: calculating a first performance score based on the first data, a second performance score based on the second data, and a third performance score based on the third data; and aggregating the first performance score, the second performance score, and the third performance score to generate the performance index value.
 3. The system of claim 1, the first data comprising at least one of a software version, an average CPU usage, an average RAM usage, or an amount of free disk space for the server.
 4. The system of claim 1, the second data comprising at least one of a status indication, a firmware version, an average memory usage, a temperature, or a battery level of the supervisory controller or the field controller.
 5. The system of claim 1, wherein the one or more edge building devices include one or more sensors or actuating devices.
 6. The system of claim 5, the third data comprising an indication of sensor alarms or sensor faults for the one or more edge building devices.
 7. The system of claim 6, the third data comprising an indication that one of the one or more actuating devices is stuck, or an indication that one of the one or more actuating devices is in a manual operating mode.
 8. The system of claim 1, wherein the third data further indicates a number of setpoints in the BMS that are overridden, and a number of edge building devices that are in a manual operation status.
 9. The system of claim 1, wherein: calculating the first performance score comprises determining a first set of performance parameters for the server, wherein a penalty score is added to the first performance score when the server does not meet one of the first set of performance parameters; calculating the second performance score comprises determining a second set of performance parameters for the supervisory controller or the field controller, wherein a penalty score is added to the first performance score when the supervisory controller or the field controller does not meet one of the second set of performance parameters; and calculating the third performance score comprises determining a third set of performance parameters for the one or more edge building devices, wherein a penalty score is added to the third performance score when one of the one or more edge building devices does not meet one of the third set of performance parameters.
 10. The system of claim 1, the operations further comprising initiating an action based on the performance index value.
 11. The system of claim 10, wherein the action comprises automatically scheduling maintenance for the server, the supervisory controller, the field controller, or at least one of the one or more edge building devices based on the performance index.
 12. The system of claim 10, wherein the action comprises transmitting a notification to a user device indicating that performance index and identifying one or more BMS components that are negatively impacting the performance index.
 13. The system of claim 1, the operations further comprising generating a recommendation for improving performance of the BMS based on the performance index.
 14. The system of claim 13, wherein the recommendation indicates one of more of the server, the supervisory controller, the field controller, or the one or more edge building devices that are negatively impacting the performance index, and wherein the recommendation further indicates one or more corrective actions to improve the performance index.
 15. A method comprising: obtaining first data indicating operating parameters of a server of a building management system (BMS); obtaining second data regarding indicating operating parameters of at least one of a supervisory controller or a field controller of the BMS; obtaining third data regarding indicating operating parameters of one or more edge building devices; and calculating a performance index value for the BMS based on an aggregate of the first data, the second data, and the third data.
 16. The method of claim 15, wherein the method further comprises: calculating a first performance score based on the first data, a second performance score based on the second data, and a third performance score based on the third data; and aggregating the first performance score, the second performance score, and the third performance score to generate the performance index value.
 17. The method of claim 15, wherein the method further comprises: calculating the first performance score comprises determining a first set of performance parameters for the server, wherein a penalty score is added to the first performance score when the server does not meet one of the first set of performance parameters; calculating the second performance score comprises determining a second set of performance parameters for the supervisory controller or the field controller, wherein a penalty score is added to the first performance score when the supervisory controller or the field controller does not meet one of the second set of performance parameters; and calculating the third performance score comprises determining a third set of performance parameters for the one or more edge building devices, wherein a penalty score is added to the third performance score when one of the one or more edge building devices does not meet one of the third set of performance parameters.
 18. A non-transitory computer-readable media comprising computer-readable instructions stored thereon that when executed by a processor cause the processor to perform operations comprising: obtaining first data indicating operating parameters of a server of a building management system (BMS); obtaining second data regarding indicating operating parameters of at least one of a supervisory controller or a field controller of the BMS; obtaining third data regarding indicating operating parameters of one or more edge building devices; and calculating a performance index value for the BMS based on an aggregate of the first data, the second data, and the third data.
 19. The non-transitory computer-readable media of claim 18, the operations further comprising: calculating a first performance score based on the first data, a second performance score based on the second data, and a third performance score based on the third data; and aggregating the first performance score, the second performance score, and the third performance score to generate the performance index value.
 20. The non-transitory computer-readable media of claim 18, the operations further comprising: calculating the first performance score comprises determining a first set of performance parameters for the server, wherein a penalty score is added to the first performance score when the server does not meet one of the first set of performance parameters; calculating the second performance score comprises determining a second set of performance parameters for the supervisory controller or the field controller, wherein a penalty score is added to the first performance score when the supervisory controller or the field controller does not meet one of the second set of performance parameters; and calculating the third performance score comprises determining a third set of performance parameters for the one or more edge building devices, wherein a penalty score is added to the third performance score when one of the one or more edge building devices does not meet one of the third set of performance parameters. 